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ABSTRACT 

This paper discusses methodological issues and 
strategies that underlie the development of expert software systems 
that provide research assistance to school practitioners of action 
research. The following key issues related to the development of 
expert systems software are examined: developing a knowledge base 
that represents research expertise; creating flexible development 
strategies and software environments; and addressing school research 
practitioners 1 special consultive needs. In conclusion, expert 
systems methodology for action research will be most effectively used 
in conjunction with an extended apprenticeship model, will 
reformulate how methodological principles are represented, and will 
improve the training of novice educational researchers. (LMI) 
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Artificial intelligence methodology offers a potential means for insuring 
"real time" accessibility ofconsuttb/e research expertise to school action 
research practitioners. Adopting perspectives based upon proven 
applications of artificial intelligence applications in industry, this paper 
discusses methodological strategies and issues that underlie the 
development of such software environments. 



Action Research as a Setting For Professional Consultation 

This paper reflects the view that action research is a type of applied research (Bogdan & Biklin, 
1992) that happens to be conducted by those whose motivation is to directly experience its potential 
benefits. Thus, in schools, action research might be conducted by principals, teachers, administrators, or 
even by students in their own interest. In this sense, we recognize that action research necessarily has a 
perspective that is specifically "local* in emphasis. Yet, because we consider the goal of research the 
establishment of knowledge between as well as within settings, our feeling is that it is inappropriate to 
limit the implications of any research endeavor purposively (particularly those from action research) that 
otherwise might be generalizable simply because of the motivations of the researchers themselves. 
Rather, we feel that as a subset of research inquiry, action research must meet all standards of research 
practice that apply whenever a study is designed to allow conclusions to be reached, whether they are of 
theoretical significance, practical utility, or both. 

Considering action research as a subset of research in general has an important utilitarian 
function in addressing the technical question of how to provide research assistance to inexperienced 
researchers, some of whom might be practitioners. This is because this assumption allows the same 
knowledge-domain that represents the core expertise of an idealized experienced researcher to serve as 
a basis for effective guidance across all research settings. This is not to say that all settings are identical, 
for they are not, but rather that they have much in common. Certainly, we recognize (along with those 
who conduct programmatic research in an area) that there are important nuances within substantively 
different areas of inquiry. Nevertheless, our concern here will be to address the question of providing 
general consultive assistance to research practitioners of limited experience. In this context, we feel it is 
sound to approach the question of providing such research assistance by assuming a common 
knowledge-base representing idealized research expertise whose availability is technologically 
engineered for researchers who have different degrees of experience and capabilities and who conduct 
research in various school settings. 



* Paper presented as part of the Symposium for School-Initiated Action Research as Environments for 

Collaborative Problem-Solving at the Annual Meeting of the American Educational Research Association, Atlanta, 
GA, April, 1993. 
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The Concept of an Expert System Consultant 

The utilization of an expert system by a school (i.e., action) researcher is equivalent to an 
interactive consultation with a highly experienced human expert to gain assistance in addressing a 
research problem (e.g., What should I do to conduct research on this topic? , What should my research 
questions be? ) or in solving a specific problem within a research study (e.g., Given the conclusions I want 
to make, how should I design my study? . Given my study, how should I measure the outcomes?). During 
such a consultation, the expert and school researcher would interact to describe the problem, clarify the 
specific details and constraints, and then specify the actions to be taken. In guiding the interaction, the 
expert's knowledge of research principles in conjunction with practical rules of thumb based upon 
extensive research experience would be linked to the school researcher's specific knowledge of the 
problem solution. As part of this process, the consultant would be able to answer questions explaining 
why certain information is needed, how various conclusions were reached, and how possible changes in 
the research setting (e.g., research questions, practical constraints) might require alternate solutions (i.e., 
what if questions). 

Within an expert systems application, school researchers would obtain similar consultive 
assistance through expert systems software developed to emulate the competence of such an 
experienced human expert. Under conditions in which the availability of human experts is limited, the 
potential value of expert systems software is its* capability to deliver consultive expertise to an individual 
with a degree of frequency or level of depth that a human expert could not provide due to limited 
accessibility. In developing such software, the major logical requirements that must be addressed 
technically are the representation of research expertise (i.e., the knowledge-base), the specification of 
knowledge-processing mechanisms that control the linking of the knowledge to the specifics of the 
problem setting in order to identify appropriate courses of action (i.e., the inference engine), and the 
design of the interface through which the school researcher and software will interact. 

An important step in the development of expert systems software is the selection of a software 
environment for building (and delivering) the system. Although any general purpose programming 
language, in principle, could be used, specially designed software development environments called 
"shells" are commonly used because they facilitate rapid prototyping and efficient system construction for 
a variety of methodological reasons. Among the most important is that, the architecture of the shells 
separates the "knowledge" component of the software from its other parts (e.g., user interface, 
management of consultive data, explanation facility, knowledge-processing/inference component). By 
using shells, developers of expert systems software become free to focus upon the encoding of expert 
knowledge that allows the software to emulate the performance of an expert rather than writing computer 
program(s) to implement all the various components of the system. Thus, the shell itself provides the 
developer with a supportive environment containing all of the tools needed to produce the system with 
the exception of the knowledge (that the developer inserts into the system) and the nature of the 
interface (about which the developer decides). 

When encoding expertise into the shell software, a common form of knowledge representation 
consists of production rules in the form of IF/THEN statements through which actions (i.e., THENs) are 
tied to patterns of conditional information (i.e., IFs). With such knowledge-based rules entered into the 
software shell, the expert system is able to decide what actions to recommend for problem-solving during 
a consultation by querying the user regarding circumstances specific to a particular setting. By matching 
patterns of circumstances to patterns of production rule conditions (IFs) and by following logical chains of 
such rules, the expert systems software can focus upon obtaining relevant information (and ignoring 
irrelevant details) while determining a proper course of action. In turn, the actions recommended by the 
program provide guidance for the end-user. 

The development of an expert system follows a general strategy for capturing human expertise 
that is commonly referred to as "knowledge engineering". First, one or more experts are carefully studied 
across a representative range of problem-solving applications. The purpose here is to determine what 
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the expert does, the conditions unae, which it is done, and <*^^^^^ 
as to whv it is done. Second, the resulting expertise is represented in IF/THEN P"***™ ™ e ^ or * 
nwe ad^ancedl knowledge representation scheme) and input into the expert system shell. JMJhi 
sTstemls ^pHeSra variety of representative problem situatb^. the results compared wrth ^ 
Smerts and the knowledgeiase (e.g., production rules) modified as necessary Fourth the revs™ } e f 

using a variety of expert system shells and a monthly trade magazine, ^Expert, regularly reviews expert 
systems software for popular microcomputer platforms (IBM. Macintosh). 

Methodological lssu«* in the Developmen t nf P^rt Systems th*t Support the Practice of Research 

This section overviews key issues specifically relating to the development of expert systems 
software that would provide research assistance to school practitioners. 

rw^nin n the Knowledae- Base that Represents R esearch Expertise Atth ° u ^^^^° n 
research methods provide an initial point for determining research expertise, they do not P ro ^ e 

cTrSuct research is far broaderthan that encompassed by traditional textbooks in "^™*"* f ln 
effect, despite their value contributing to the knowledge-base in some areas (ej^ analyse of 
date aspects of experimental design, aspects of measurement procedures), textbooks as a whole are 
SmpteTreprerent^ of the broJlange of expertise required to conduct ^ research 
Because textbook-based knowledge is of limited value, the full development* a research 

knowledge-base can only result from the analysis of the P^^SLX^nS Presented a 
representative range of research problems. With this need in mind, Wale (1993) haspresented a 
scheme for classifying empirical research studies that could provide the basis for specifying the 
SlSI by reflecting the stages through ^^^^^A^ 
The resulting continuum then could provide sufficient structure m the form of a domain to facilitate the 
explication of categories of research expertise that focus on the following: 

* establishing the existence of a phenomenon 

determining the conditions under which a phenomenon occurs 

* relating the phenomenon to other phenomena 

establishing manipulatable conditions that determine the occurrence of the phenomenon 
developing theory in the form of the symbolic codification of a through d 

engineering technological applications that incorporate theory (considered as scientific 
knowledge) 

verifying the soundness of technological applications 
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Complementing this scheme, Vitale (1993) argued for mapping the interrelationship of various 
types of research perspectives (e.g., qualitative, quantitative, applied, theoretical, action, etc.) in terms of 
a two underlying elements: (a) an absolute distinction between experimental and non-experimental 
research in combination with (b) a continuum of research settings ranging from artificial (or laboratory) vs 
naturalistic. 

From the standpoint of organizing research problems (and expertise), the idea that either 
experimental or non-experimental research (considered to allow qualitatively different forms of 
conclusions) may be conducted in either naturalistic or laboratory settings and the idea that the domain of 
al! research (including the scheme above) is encompassed by these two classification dimensions has 
much to recommend it with regard to parsimony. In this spirit (after Sidman, 1960), this scheme 
considers all research as theoretical, with some theoretical research also being applied if it addresses 
behaviors having societal value; while (after Johnson & Pennypacker, 1980), the concerns of qualitative 
researchers are repositioned to fall within a broadened perspective having to do with measurement and 
scaling. In particular, from the standpoint of addressing the question of explicating the research expertise 
required for action research, Vitale's (1993) scheme implies that qualitative and quantitative research 
may be represented as part of a broader research perspective, and, thus, do not represent fundamentally 
different distinctions that would require diferent domains of expertise. Rather, this distinction is 
considered to denote sociological characteristics of researchers that are not encompassed within 
expertise in combination with methodological preferences (and styles) that can be incorporated into a 
common expertise domain, methodological styles that would be represented in expertise. In any case, 
whether Vitale's (1993) or an alternate scheme is used to represent the domain of research practice, the 
performance of expert researchers across the domain must be represented fully in the knowledge-base 
of the expert system in order for it to provide effective guidance through consultation. In this regard, 
textbooks on research methods are clearly inadequate in both breath and depth and some form of 
knowledge engineering strategies are necessarily required. 

Flexible Development Strategies and Software Environments . Although the process for 
developing an expert system in general and the knowledge-base in particular has been presented as if it 
consists of a sequential series of discrete steps, in most cases it is not. Rather, the development of an 
expert system is much more of a "bootstrap 14 operation in which different parts of the development 
process interact with one another. This is because the knowledge of most experts consists of what are 
known as "ill-defined domains." As a result of what might be considered true" expertise being very 
difficult to capture directly, the development of most expert systems emphasizes incremental prototyping 
(vs top-down design) as a development strategy in which increasingly larger parts of the system are 
implemented and evaluated until the final system is completed. 

In following a prototyping strategy, the selection of a software development environment (e.g., 
shell) is critical since extensive editing and restructuring of the knowtedge-base may be required at any 
point in the development process and since the size of the system cannot be determined in advance. 
Therefore, before initiating a project using a shell, it is important to evaluate the software environment in 
terms of these concerns and, if possible, talk directly to individuals (i.e., experts) who have prior 
experience developing the type of application that is being considered regarding these and other * 
technical concerns (e.g., type of inference engine(s) supported, object-oriented knowledge strictures). 
Although beyond the scop* of this paper, some good expert systems shells to investigate for IBM 
microcomputer platforms running under Microsoft Windows 3.0 or above are Symbologic Adept , 
Goldworks III , Level 5 Object , and EXSYSS . 

In many cases, the application domain, the associated expertise, and the needs of the end-user 
are so ill-defined that substantial preliminary development work must be done in order to clarify the 
consultive problem being addressed. Our feeling is that the question of providing consultive assistance 
to practitioners is clearly in this category. As a result, our present efforts in developing consultive 
research systems are pursuing the following strategy: 



ERLC 



5 



Al & Action Research 
Pages 



* an initial attempt to frame-out the domain of research and research expertise in a 
parsimonious and modular form 

an explication of the consultive needs of the school researcher having little research 
experience 

the use of a hypertext-type of software environment (e.g., Toolbook on the IBM platform) 
that allows some aspects of the potential knowledge base and the means of 
communication of information between software and user to be explored 

the initial prototyping of modular elements of the expert system using Symboiogic Adept 
which provides unique visual programming environment in which procedures as 
sequences of activities can be represented directly 

the subsequent development of the system using a knowledge-based expert system 
(e.g., Goldworks III) that supports production rales as part of more advanced forms of 
integrated knowledge structures (e.g., frames, objects) and alternate forms of inference 
(e.g., backward chaining, forward chaining, hybrid models). 

Overall, the adoption of the above strategy allowing for the use of a number of complementary 
software environments was bassd upon a number of considerations. First, a major underlying concern 
was the assumption that the explication of the research knowledge-base in conjunction with the needs of 
the school practitioner would likely involve substantial revision. This is simply due to the fact not all 
aspects of educational research methodology are well developed and the many areas that are 
well-developed do not always represent methodology in the form of the capabilities or procedures of 
which expertise consists. Second, some interactive field-testing is required to develop a workable 
interface model. This is far easier to do in a limited prototyping fashion using a hypertext environment 
such as Toolbook, before encoding complex forms of knowledge into an expert system shell. 

The third consideration is important because we anticipate that an emphasis on modularity and 
the capability to integrate different software environments may facilitate the development effort and 
provide a more naturally-usable form of consultive software for school researchers. For example, during 
a consultation it might be natural for the user to browse and select different alternatives- a capability that 
is much easier to develop in a hypertext environment such as Toolbook. Or, some procedures might be 
developed most easily in Symboiogic Adept because it is a form of shell that is designed to allow 
procedural knowledge to be naturally encoded. Or, more complex decisions involving the evaluation of 
sets of rules might be more readily developed through prototyping in a knowledge-based environment 
such as Goldworks III. In all cases, because each of the three environments can be integrated with the 
others, it is possible to integrate components of the overall system that have been developed through 
prototyping using different software environments. With this capability in mind, it is possible for an 
underlying prototyping strategy to be implemented efficiency. 

Some Categories of Special Consultive Needs for School Research Practitioners . In identifying 
priorities associated with developing a consultive system, our particular interest is in focusing our efforts 
on the elements of the research process that experienced researchers emphasize (and which novices 
would be unaware) rather than those emphasized in textbooks. Although beyond the scope of this 
paper, some of these address broader aspects of research as inquiry such as (a) planning and 
implementing programmatic research (vs a single-study perspective), (b) strategies for networking with 
researchers of similar interests, (c) methodological approaches to qualitatively analyzing (e.g., observing, 
measuring, describing) both problem settings and treatment effects, (d) procedures for evaluating trends 
in research literatures and identifying (in specific areas) parallel research developments by 
non-educational researchers, and (e) standards for designing research given intended conclusions and 
for interpreting conclusions given design. Although in the formative stages, we feel that it is possible to 
develop expert systems software that provides at least minimal guidance for inexperienced researchers 
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in modular forms that could eventually be encompassed within a single program. At the same time, we 
also envision working to develop applications that support the more traditional research domains such as 
test development, statistical analysis, and traditional research design. 

Implications of Expert Systems Methodology for Action Research 

There are a number of important implications that follow from the application of expert systems 
technology to the problem of providing research support to school practitioners conducting action 
research. These are; 

* The best use of such a system (and the best environment for development) would be in 
conjunction with the type of extended apprenticeship model described by Romance & 
Vitale (1 993) in this symposium. Within this context, a major goal would be to capture 
different aspects of research expertise across the range of possible research 
applications as they are being supported. Although we can envision a useful collection 
of "stand-alone" expert systems emphasizing different aspects of the research process in 
a limited fashion, the overall concept of capturing all research expertise within a single 
software application having great levels of detail is unrealistic for the immediate future. 
Thus, we believe it is presently feasible to develop limited expert systems that school 
action researchers would find useful. 

It is clear to us that any serious attempt to represent expertise within the field of research 
would eventually lead to an improved reformulation of how methodological principles in 
research are represented. At the present time, just the application of par^mony alone 
would help eliminate the excessive terminology that professionals new to the field must 
face. However, even beyond verbal streamlining, any serious attempt to address the 
question of research methodology as procedural rather than declarative knowledge 
would have the effect of refocusing attention upon what the purpose of research is within 
the field and how it should be conducted. 

Finally, it is also clear that having the capability to develop knowledge-based systems 
that provide expert consultation would also have an effect on how researchers can be 
trained in general. Specifically, having the capability to represent research expertise 
across the domain of research practice could directly support the development of 
instructional models (e.g., Lawler, 1987) from the field of artificial intelligence, including 
intelligent (i.e., knowledge-based) tutoring systems in which students would receive 
computer-based guidance as they apply methodological skills to solve problems or 
engage in simulations to gain higher order research capabilities. In effect, the 
development of expert systems, even limited ones, would open the gateway for dramatic 
improvements in training novice educational researchers, both in and out of school 
settings. 
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